Implementation of Practical Surface SARS-CoV-2 Surveillance in School Settings

ABSTRACT Surface sampling for SARS-CoV-2 RNA detection has shown considerable promise to detect exposure of built environments to infected individuals shedding virus who would not otherwise be detected. Here, we compare two popular sampling media (VTM and SDS) and two popular workflows (Thermo and PerkinElmer) for implementation of a surface sampling program suitable for environmental monitoring in public schools. We find that the SDS/Thermo pipeline shows superior sensitivity and specificity, but that the VTM/PerkinElmer pipeline is still sufficient to support surface surveillance in any indoor setting with stable cohorts of occupants (e.g., schools, prisons, group homes, etc.) and may be used to leverage existing investments in infrastructure. IMPORTANCE The ongoing COVID-19 pandemic has claimed the lives of over 5 million people worldwide. Due to high density occupancy of indoor spaces for prolonged periods of time, schools are often of concern for transmission, leading to widespread school closings to combat pandemic spread when cases rise. Since pediatric clinical testing is expensive and difficult from a consent perspective, we have deployed surface sampling in SASEA (Safer at School Early Alert), which allows for detection of SARS-CoV-2 from surfaces within a classroom. In this previous work, we developed a high-throughput method which requires robotic automation and specific reagents that are often not available for public health laboratories such as the San Diego County Public Health Laboratory (SDPHL). Therefore, we benchmarked our method (Thermo pipeline) against SDPHL’s (PerkinElmer) more widely used method for the detection and prediction of SARS-CoV-2 exposure. While our method shows superior sensitivity (false-negative rate of 9% versus 27% for SDPHL), the SDPHL pipeline is sufficient to support surface surveillance in indoor settings. These findings are important since they show that existing investments in infrastructure can be leveraged to slow the spread of SARS-CoV-2 not in just the classroom but also in prisons, nursing homes, and other high-risk, indoor settings.


Preparing Revision Guidelines
To submit your modified manuscript, log onto the eJP submission site at https://msystems.msubmit.net/cgi-bin/main.plex. Go to Author Tasks and click the appropriate manuscript title to begin the revision process. The information that you entered when you first submitted the paper will be displayed. Please update the information as necessary. Here are a few examples of required updates that authors must address: • Point-by-point responses to the issues raised by the reviewers in a file named "Response to Reviewers," NOT IN YOUR COVER LETTER. • Upload a compare copy of the manuscript (without figures) as a "Marked-Up Manuscript" file. • Each figure must be uploaded as a separate file, and any multipanel figures must be assembled into one file. ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data. If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record. If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication of your article may be delayed; please contact the ASM production staff immediately with the expected release date.
For complete guidelines on revision requirements, please see the journal Submission and Review Process requirements at https://journals.asm.org/journal/mSystems/submission-review-process. Submission of a paper that does not conform to mSystems guidelines will delay acceptance of your manuscript.
This study compared two workflows (SDS/Thermo and VTM/PE) for surface surveillance of SARS-CoV-2 virus, using both contrived and realistic samples. This study is valuable to combat pandemic spread and keep people in indoor public settings safe. While the study is beneficial, it lacks important explanations so that the research outcomes can be readily leveraged.
For example, the manuscript claimed that the authors' method shows superior sensitivity and specificity than the SDPHL pipeline. While I can tell from the manuscript that the SDS/Thermo pipeline is more sensitive, I do not see the authors discuss and compare the specificity between the two pipelines. Adding contents about specificity may be necessary.
Discussions on testing using realistic samples are generally solid. However, the strength of the arguments needs to be improved. 1) More explicit references to figures are needed to better demonstrate the connection between evidence and interpretation. 2) Elaborations are needed for a few arguments. Specifically, L147-156 in general: Please refer to the specific panel in figure 3 to back up the interpretations.
L147-148: Specify the conclusion that each laboratory performed best when sample extraction and RT-qPCR processing occurred in the same facility is in terms of the number of positive samples (Fig. 3D), average Cq ( Fig. 3A&B?), or something else. L149-152: 1) If I understand it correctly, this was interpreted from Fig. 3C. If so, clearly refer to Fig. 3C so that no one will be confused. 2) It will be better to mark out where the mean Cq is for each group in Fig. 3C (maybe using a short horizontal line). Right now, it is unclear whether the SDPHL RT-qPCR assay performed better on the VTM swabs or on the SDS swabs by just looking at the figure. 3) Both Cq differences are not statistically significant. Why is 1.2 substantially better while 0.88 only slightly better? L152-154 (However…Fig. 3): 1) What is the comparison group that leads to the conclusion of the substantial advantage in sensitivity of 5.1 Cq units? VTM/PE combination? 2) Please elaborate on the statistical testing done here and in Fig. 3C. Are the two (Kruskal-Wallis, p < 0.01) results of the same testing? In L152-154, if the conclusion is drawn by comparing SDS/Thermo with VTM/PE, post-hoc tests should be performed following Kruskal-Wallis. In Fig. 3C, are samples extracted in both UCSD and PHL treated as the same group? In other words, is the testing performed based on four groups or two? If the former, post-hoc tests should be conducted as well. And the significance representation may need to be modified. If the latter, nonparametric equivalent of t test is more appropriate, e.g., Wilcoxon Test. Figure 3C: The significance representation is confusing. By convention, one asterisk (*) means 0.01 < p ≤ 0.05, but the caption clearly mentioned p < 0.01. L160-162: Please explain why sampling twice can compensate for the sensitivity gap between VTM/PE assay and SDS/Thermo assay.
Additionally, since this comparison study targets practical implementation of the pipelines, I have the following suggestions to improve its clarity.

Uniform the wording
After reading through the manuscript, I understand that there are two workflows and in the crossover comparison, RT-qPCR assays were exchanged while swabbing/transport medium and RNA extraction remain unchanged.

Lab
Sample collection RNA extraction RT-qPCR UCSD SDS Thermo pipeline Thermo pipeline SDPHL VTM Perkin-Elmer pipeline Perkin-Elmer pipeline This is clear when it was first introduced in L141-145. But downstream, the inconsistency of wording adds confusion. I suggest defining the wording first and sticking to the defined terms in the rest of the manuscript. For example, as per the current manuscript, confusion might occur in the following instances.
• In L149-150, the term "VTM swabs" actually represent samples collected in VTM and extracted following the Perkin-Elmer pipeline, while according to L114-125, RNA extraction step reads like part of the processing pipeline rather than falling under the name of swabbing/transport medium. Similarly, do the VTM and SDS in Supplementary Table S1 also represent from sample collection to RNA extraction?
• In Figure 3 and Supplementary Table S1, lab names/locations (UCSD, PHL) were used to represent the pipeline while in Results, the pipeline names (SDS/Thermo, VTM/PE) and lab names seemed to be used interchangeably. Furthermore, wording of lab names was not standardized, e.g., PHL in Figure 3 and County in Supplementary Table S1.

Other comments:
L307: Recommend using p = 2.87 x 10 -6 rather than p < 0.01 in the figure caption to match what is used in L132-133.  Figure 2: The idea of using a heatmap to represent the counts of positive detection events is great. But the color differences among 3, 2, and 1 are not large enough for easy distinguishment. Try to make it clearer. Maybe switching to a 2-color scale would help? Figure 3: The sequential color scheme to indicate the number of gene targets is hard to distinguish, especially for PHL. Figure 3 A&B: 1) It would be more visually friendly to order the x axis consistently across A and B for sampling locations (i.e., sample names). 2) I suggest including the sample names in x axis for those with no detection and using some type of label to indicate that they are not detected so that the readers are better informed.
For protocol clarity, please include the following information in the UCSD protocol. We thank the reviewer for their time and for carefully reviewing our manuscript. We believe their comments have led to us seeking out references that greatly improved the strength of the paper. We have taken their constructive comments and edited our manuscript appropriately.

Summary/Overview:
This study compared two workflows (SDS/Thermo and VTM/PE) for surface surveillance of SARS-CoV-2 virus, using both contrived and realistic samples. This study is valuable to combat pandemic spread and keep people in indoor public settings safe. While the study is beneficial, it lacks important explanations so that the research outcomes can be readily leveraged.
2 For example, the manuscript claimed that the authors' method shows superior sensitivity and specificity than the SDPHL pipeline. While I can tell from the manuscript that the SDS/Thermo pipeline is more sensitive, I do not see the authors discuss and compare the specificity between the two pipelines. Adding contents about specificity may be necessary.
We appreciate this suggestion and clarified the sensitivity of our pipeline on lines 162-168.
Discussions on testing using realistic samples are generally solid. However, the strength of the arguments needs to be improved. 1) More explicit references to figures are needed to better demonstrate the connection between evidence and interpretation. 2) Elaborations are needed for a few arguments. Specifically, L147-156 in general: Please refer to the specific panel in figure 3 to back up the interpretations.
Thank you for this suggestion. We have referenced the specific panels in Figure 3 (A and B), which provide support for our interpretations, as suggested (line 148).
L147-148: Specify the conclusion that each laboratory performed best when sample extraction and RT-qPCR processing occurred in the same facility is in terms of the number of positive samples (Fig. 3D), average Cq (Fig. 3A&B?), or something else.
We appreciate this suggestion, and we have incorporated the reference to Figure 3C (line 149), which this statement is based on (although the other panels mentioned also support it).
L149-152: 1) If I understand it correctly, this was interpreted from Fig. 3C. If so, clearly refer to Fig. 3C so that no one will be confused.
We appreciate this feedback and have incorporated the reference to Figure 3C as suggested (line 149).
2) It will be better to mark out where the mean Cq is for each group in Fig. 3C (maybe using a short horizontal line). Right now, it is unclear whether the SDPHL RT-qPCR assay performed better on the VTM swabs or on the SDS swabs by just looking at the figure. 3) Both Cq differences are not statistically significant. Why is 1.2 substantially better while 0.88 only slightly better?
We have added a short horizontal line to each group in Fig. 3C as suggested. We were informally using the threshold of 1 Cq unit for "substantially" versus "slightly", but agree that this is not well justified, so we have deleted "substantially" and "slightly" and simply report them as better (lines 152-156).
L152-154 (However…Fig. 3): 1) What is the comparison group that leads to the conclusion of the substantial advantage in sensitivity of 5.1 Cq units? VTM/PE combination? 2) Please elaborate on the statistical testing done here and in Fig. 3C. Are the two (Kruskal-Wallis, p < 0.01) results of the same testing? In L152-154, if the conclusion is drawn by comparing 3 SDS/Thermo with VTM/PE, post-hoc tests should be performed following Kruskal-Wallis. In Fig. 3C, are samples extracted in both UCSD and PHL treated as the same group? In other words, is the testing performed based on four groups or two? If the former, post-hoc tests should be conducted as well. And the significance representation may need to be modified. If the latter, nonparametric equivalent of t test is more appropriate, e.g., Wilcoxon Test.
We appreciate the reviewer's detailed attention to the statistical analysis. We have now clarified that the comparison group that leads to the advantage of 5.1 Cq units is indeed drawn by comparing the Thermo and PE workflows as the reviewer appreciates (lines 144-146). We have added a note to clarify this. The Kruskal-Wallis test was performed across all four combinations and was found to be significant. Performing a post-hoc test comparing each combination of groups directly, via Wilcoxon Rank-Sum, yields a P-value of 5.86 x 10 -3 for the Thermo vs PE pipelines and we have added this to the manuscript on line 158. We have also added in the Wilcoxon Rank-Sum results for the other combinations on lines 148-159. We have clarified our statistical analysis in the caption of Figure 3C.
Results of Wilcoxon Rank-Sum pairwise Figure 3C: The significance representation is confusing. By convention, one asterisk (*) means 0.01 < p ≤ 0.05, but the caption clearly mentioned p < 0.01 We apologize for the error and have fixed the inconsistency.
L160-162: Please explain why sampling twice can compensate for the sensitivity gap between VTM/PE assay and SDS/Thermo assay. 4 On further analysis, we agree that this may not solve the issue, depending on whether errors that lead to dropout are stochastic or deterministic, and have deleted this suggestion accordingly.
Additionally, since this comparison study targets practical implementation of the pipelines, I have the following suggestions to improve its clarity.

Uniform the wording
After reading through the manuscript, I understand that there are two workflows and in the crossover comparison, RT-qPCR assays were exchanged while swabbing/transport medium and RNA extraction remain unchanged.

Lab
Sample collection RNA extraction RT-qPCR UCSD SDS Thermo pipeline Thermo pipeline SDPHL VTM Perkin-Elmer pipeline Perkin-Elmer pipeline This is clear when it was first introduced in L141-145. But downstream, the inconsistency of wording adds confusion. I suggest defining the wording first and sticking to the defined terms in the rest of the manuscript. For example, as per the current manuscript, confusion might occur in the following instances.
• In L149-150, the term "VTM swabs" actually represent samples collected in VTM and extracted following the Perkin-Elmer pipeline, while according to L114-125, RNA extraction step reads like part of the processing pipeline rather than falling under the name of swabbing/transport medium. Similarly, do the VTM and SDS in Supplementary Table S1 also represent from sample collection to RNA extraction?
• In Figure 3 and Supplementary Table S1, lab names/locations (UCSD, PHL) were used to represent the pipeline while in Results, the pipeline names (SDS/Thermo, VTM/PE) and lab names seemed to be used interchangeably. Furthermore, wording of lab names was not standardized, e.g., PHL in Figure 3 and County in Supplementary Table S1.
We thank the reviewer for catching these inconsistencies and have corrected them as recommended.

Other comments:
L307: Recommend using p = 2.87 x 10 -6 rather than p < 0.01 in the figure caption to match 5 what is used in L132-133.
Thank you for pointing this out. The specific p-value has been added to the figure caption.  Figure 2: The idea of using a heatmap to represent the counts of positive detection events is great. But the color differences among 3, 2, and 1 are not large enough for easy distinguishment. Try to make it clearer. Maybe switching to a 2-color scale would help?
We have switched to a 2-color scale as suggested. Figure 3: The sequential color scheme to indicate the number of gene targets is hard to distinguish, especially for PHL.
We have switched to a color scheme that highlights differences between adjacent bands rather than the sequential color scheme used. Figure 3 A&B: 1) It would be more visually friendly to order the x axis consistently across A and B for sampling locations (i.e., sample names). 2) I suggest including the sample names in x axis for those with no detection and using some type of label to indicate that they are not detected so that the readers are better informed. Congratulations! Your manuscript has been accepted, and I am forwarding it to the ASM Journals Department for publication. For your reference, ASM Journals' address is given below. Before it can be scheduled for publication, your manuscript will be checked by the mSystems production staff to make sure that all elements meet the technical requirements for publication. They will contact you if anything needs to be revised before copyediting and production can begin. Otherwise, you will be notified when your proofs are ready to be viewed.
ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data. If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record. If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication of your article may be delayed; please contact the ASM production staff immediately with the expected release date.
As an open-access publication, mSystems receives no financial support from paid subscriptions and depends on authors' prompt payment of publication fees as soon as their articles are accepted.

Publication Fees:
You will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
Corresponding authors may join or renew ASM membership to obtain discounts on publication fees. Need to upgrade your membership level? Please contact Customer Service at Service@asmusa.org.
For mSystems research articles, you are welcome to submit a short author video for your recently accepted paper. Videos are normally 1 minute long and are a great opportunity for junior authors to get greater exposure. Importantly, this video will not hold up the publication of your paper, and you can submit it at any time.
Details of the video are: · Minimum resolution of 1280 x 720 · .mov or .mp4. video format · Provide video in the highest quality possible, but do not exceed 1080p · Provide a still/profile picture that is 640 (w) x 720 (h) max · Provide the script that was used We recognize that the video files can become quite large, and so to avoid quality loss ASM suggests sending the video file via https://www.wetransfer.com/. When you have a final version of the video and the still ready to share, please send it to mSystems staff at msystems@asmusa.org.
For mSystems research articles, if you would like to submit an image for consideration as the Featured Image for an issue, please contact mSystems staff at msystems@asmusa.org.
Thank you for submitting your paper to mSystems.  Table S3: Accept Table S1: Accept Table S2: Accept